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Evaluating a School-based Program 2 

ABSTRACT 

Differences in the effectiveness of a school-based prevention-with-intervention and a 
prevention-only program for aggressive children were examined. A total of 32 teacher- 
referred children, matched on externalizing behavior, age, and sex, were randomly assigned 
to either a treatment group which received both the prevention and intervention components 
in Semester 1 or a waiting list group which received only the prevention components in 
Semester 1. Subsequently, in Semester 2, the waiting list group received both the 
prevention and intervention components. Results indicated that the treatment group was 
rated by their teachers as improved at posttreatment, relative to the waiting list group, and 
was not different than a nonclinical control group on several subscales of a shortened 
version of the Teacher Report Form (TRF; Achenbach & Edelbrock, 1986). These 
treatment gains were not maintained at followup, however, nor did the waiting list group 
show any treatment effects after receiving the intervention. Factors accounting for these 
unexpected results are discussed. 
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INTRODUCTION 

This past decade has seen an increase in the incidence and severity of violent and aggressive 
behavior among children. According to the Ontario Child Health Study (OCKS; Offord, 
Alder, & Boyle, 1986), children attending schools in "high-risk" areas of the city, 
characterized by subsidized housing and low income, are the most at risk for conduct 
problems. Given the high prevalence of conduct problems and the difficulty and expense of 
treating established cases, Loeber (1990) noted that it is important to find effective 
prevention and early intervention programs. 

Schools are well suited as sites for mental health intervention because they provide 
convenient access to representative samples of children with problem behaviors. As well, 
generalization and maintenance of treatment effects are facilitated by treatment taking place 
in the setting where the child experiences difficulties and the peer group can also be 
included (Coie & Krehbiel Koeppl, 1990; Coie, Underwood, & Lochman, 1992). 

School-based interventions usually take one of two forms: prevention with an essentially 
nonidentified, at risk group (e.g., Hiebert, Kirby, & Jaknavorian, 1989; Olweus, 1992) or 
direct intervention with a targeted sample (e.g., Coie et al., 1992 Dubow, Huesmann, & 
Eron, 1987; Mize & Ladd, 1990; Schneider, 1991). However, it has also been suggested 
that the most efficacious program would combine both prevention and intervention 
components (Weissberg & Allen, 1986). 

The main focus of this study was to examine the relative effectiveness of a school- 
based prevention-only program with an intervention-with-prevention program for teacher- 
identified aggressive children who live in a community that places them at risk for the 
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development of conduct problem behaviors. A second focus of this study was to examine 
the social validity of the treatment effects with the inclusion of a nonclinical, no-treatment 
control group. 

METHOD 

Subjects and Procedure . Teachers were asked to refer children between the ages of 6 to 12 
years who presented as disruptive/aggressive to the Earlscourt School-based Program (ESP). 
A total of 32 children who met the basic criterion of a T-score equal to or greater than 60 
on the Externalizing scale of a shortened version of the Teacher's Report Form (TRF; 
Achenbach & Edelbrock, 1986) were admitted and received a more extensive assessment 
battery after parents' permission was obtained. They were then matched on conduct 
problems, age, and sex, and randomly assigned to either the treatment group (n=16) which 
received the intervention and prevention components in the first semester or the waiting list 
controi group (n = 16) which received only the prevention components in the first semester, 
and, subsequently, received both the intervention and prevention components in the second 
semester. A nonclinical control group (n = 13), rated by their teachers as having a T-score 
less than 60 on the Externalizing scale of the TRF and matched on age and sex, was used to 
assess the social validity of the program's effects. The average age of the children by group 
was 8.4 (SD = 1.5), 9.2 (£0=2.1), and 8.8 (SD=1.8) years for the treatment, waiting list, 
and control groups, respectively, (F< 1). 

Irmtment. The Earlscourt School-Based Programme (ESP) is a multifaceted intervention 
which incorporates both prevention and intensive intervention components to address the 
needs of high-risk, school children. Prevention components are offered school-wide to 
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Evaluating a School -based Program 5 
create a more positive school culture and to prevent the escalation of behavior problems. 
Prevention components included: (a) social skills training in the classroom and (b) prosocial 
theme weeks. ESP staff and teachers co-lead half-hour, weekly social skills training 
sessions. Prosocial theme weeks highlight and reinforce prosocial behavior through special 
activities and integration of the theme into regular curriculum instruction. 

Intervention components are designed to decrease aggressive, antisocial behavior and 
increase prosocial behavior in targeted children. These components included: (a) social- 
cognitive skills training groups, (b) individual coaching, and (c) family outreach. Group 
meetings lasting 75 minutes occurred twice each week for 12 weeks. Concurrent half-hour 
individual sessions were held weekly to individualize treatment to each child's particular 
needs. The nature of the family outreach varied from information-sharing about the child's 
progress to family counselling. The targeted children also benefit from the school-wide 
activities by facilitating the genera^ :ion and maintenance of their newly learned prosocial 
skills. 

Measures . Measures for the study were intended to elicit information from multiple sources 
using a variety of methods (see Table 1). Measures were collected at Time 1 (before either 
group had received the program) and at Time 2 (after the treatment group received the 
intervention and before the waiting list received the treatment). As well, the TRF was re- 
administered at Time 3 (for a three-month followup of those children in the treatment group 
and at the end of the intervention for the waiting list group). No measures were collected 
from the control group at Time 3. Lastly, the Olweus (1992) self-report bullying 
questionnaire was administered to 106 children in grades 4 to 8 in October and in June. 
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Insert Table 1 about here 



Hypotheses . 

• The treatment group will show improvement after receiving treatment. 

• The treatment group will differ from the waiting list control group at Time 2. 

• The treatment group will not differ from the control group at Time 2. 

• The waiting list group will show improvement at Time 3. 

• The waiting list group will not differ from the treatment group at Time 3. 

• The treatment group will maintain treatment gains at Time 3. 

RESULTS 

To compare the relative effectiveness of an intervention-with-prevention program and a 
prevention-only program, ? eries of 3 (group) X 2 (time) multivariate repeated measures 
analyses of variance (MANOVA) was conducted. Planned comparisons (Dunn's multiple 
comparison procedure) were subsequently conducted to test the specific hypotheses. Due to 
a loss of subjects (two children left the school before the Time 2 measures were collected) 
these analyses were conducted on n = 16 in the treatment group, n = 15 in the waiting list 
group, and n = 12 in the control group. 

Significant group X time effects were found on the Aggressive, F(2,40)=7.9, £<.001, 
Delinquent, F(2,40)=7.5, p<.002, and Externalizing, F(2,40)=6.9, p<.003, scales of the 
TRF (Achenbach & Edelbrock, 1986) and the Social Skills scale of the ACTeRS, 
F(2,40)=4.5, p<.02, (Ullmann, Sleater, & Sprague, 1987) (see Figures 1-4). Planned 
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comparisons indicated that teachers rated: (a) children in the treatment group as significantly 
improved following the program; (b) children in the treatment group as having fewer 
behavior problems and more social skills than those in the waiting list group at Time 2; and 
(c) children in the treatment group as not different than the control group on the Aggressive 
and Delinquent subscales of the TRF at Time 2. 



Insert Figures 1-4 about here 



To examine followup effects for the treatment group and treatment effects for the 
waiting list group, 2 (group) X 3 (time) MANOVAs with Dunn's multiple comparison 
procedure were conducted. Again, due to one child leaving school before the followup 
measure was collected, these analyses were conducted on n = 15 in the treatment group and 
n = 15 in the waiting list group. 

As shown in Figures 1-3, (a) the waiting list group showed no significant differences 
between Times 2 and 3 and (b) the treatment group differed significantly from the waiting 
list group at Time 3. These results suggest that the waiting list group showed no treatment 
effects after receiving the program. Moreover, the gains made by the treatment group at 
Time 2 were maintained at Time 3 only for the Delinquent subscale of the TRF. 

Lastly, a 2 (group) X 2 (time) ANOVA on the self-reported bullying questionnaire 
yielded a marginally significant interaction effect, F(l,104)=3.6, p< .06. This result 
indicated that the children who received the program (in either semester) and who completed 
the questionnaire (n^) reported significantly less bullying behavior after the program 
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(M=l-2) than before QA = \.T), while the children not identified for the program who 
completed the survey (n=97) reported no change from pretreatment to posttreatment 
(M=-68 versus M = .68). 

DISCUSSION 

The results of this study indicated that the ESP was effective in decreasing the problem 
behaviors and enhancing the social skills of the treatment group. Indeed, comparisons with 
a group of nonaggressive children indicated that the treatment group's Aggressive and 
Delinquent TRF scores were not significantly different. These findings support the clinical 
utility of this multifaceted approach to treatment with this population. At the same time, 
however, for two of the three TRF scales, these treatment gains were not maintained at 
followup. This may be due to the loss of one subject from whom followup data were not 
collected, resulting in a higher pretreatment mean score. 

The prevention-only program, while advocated as a cost-effective approach to 
intervention with high-risk populations, did not appear to be effective. In fact, the waiting 
list group showed more problem behaviors after receiving this facet of the program than 
before. Moreover, the waiting list group did not show improvements after receiving all the 
program components. This finding may reflect several things. First, teachers were not 
blind to the conditions of the children. Indeed, teachers reported experiencing difficulties 
with the waiting list group in the first semester (when they did not receive the intervention) 
resulting in a possible negative bias that was maintained for the duration of the study. The 
difficulties teachers experienced with this group may have also exacerbated the children's 
conduct problem behaviors resulting in higher pretreatment baseline scores than the 
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treatment group. Third, in spite of the matching procedure across conditions and no mean 
differences in age, the waiting list group had more younger (age 6 years) and older (age 12 
years) children than the treatment group. Given the program's cognitive based approach, it 
may have been too advanced for the younger children (Kazdin, 1993), while the older 
children may have been more difficult to engage. 

In general, while use of a matched, randomized research design with a nonclinical 
control group indicated program effectiveness with the prevention- with-intervention program, 
problems with various problems with this applied research study were encountered. These 
included a small sample size, treatment conditions to which teachers were not blind, and a 
reliance or. teachers' reports for outcome measures. Although alternative sources for data 
were used in the present study (e.g., classroom observations and self-reports), they were not 
found to be sensitive to change. Nonetheless, the positive findings we obtained warrant 
further investigation to examine which treatment components were the most effective. 
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Table 1. 

Measures Used to Evaluate the ESP 

• Shortened version of the Teacher Report Form (TRF; Achenbach & Edelbrock, 1986) 

• Shortened version of the Taxonomy of Problematic Social Situations for Children 
(TOPS; Dodge et al., 1985) 

• Social skills items (n=7) of the ADD-H Comprehensive Teacher's Rating Scale 
(ACTeRS; Ullmann et al., 1987) 

• Self-perception Scale for Children (Harter, 1985) or the Pictorial Scale of Perceived 
Competence and Social Acceptance for Young Children (Harter & Pike, 1983) 

• Classroom observations of on/off task behavior 

• Bullying survey administered school-wide (Olweus, 1992) 
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